System and method for light source calibration

ABSTRACT

A light source calibration target has a surface in view of a camera of an image capture system, The target includes a substrate having a substantially Lambertian surface, a visually distinct polygonal shape having corners formed on said surface so as to be visually distinct from the surface, and a plurality of objects each having an upstanding tip mounted on the surface, A method of using the calibration target operates, for each of the light sources to be calibrated, to capture an image of the target; process the captured image to derive light source calibration data and to store the calibration data.

TECHNICAL FIELD

These teachings relate generally to image capture systems and, morespecifically, to light sources and techniques for light sourcecalibration for image capture systems.

BACKGROUND

Advanced three-dimensional (3D) scanning systems intended for 3Dgraphics applications, and in particular for realistic renderingapplications, capture both the shape (geometric properties) andphotometric properties of objects. Machine vision systems for inspectingsurface finish quality also capture both the geometric and photometricproperties of objects.

As employed herein, capturing the “shape” of an object means to modelmathematically the 3D space occupied by the object, while capturing the“photometric properties” of an object means to model mathematically howthe object interacts with light, such as how the object reflects andrefracts light.

There exist in the literature several predictive models for lightreflection, with different levels of complexity depending on the shapeand on the type of considered object material. The data required to“fit” such models is composed of images acquired under different {yetknown} illumination conditions. In practice, the shape and photometricdata may be obtained by using several light sources of relatively smallsize, positioned at some distance from the object and preferablysomewhat evenly distributed around it. A representative example of onesuch object illumination and image capture system 10 is shown in FIG. 1.Reference may also be had to FIG. 9 of commonly assigned U.S. Pat. No.6,455,835 B1, “System, Method, and Program Product for AcquiringAccurate Object Silhouettes for Shape Recovery”, By Fausto Bernardini,Henning Biermann, Holly E. Rushmeier, Silvio Savarese and Gabriel Taubin(incorporated by reference herein), for showing an array of M (e.g.,nine) light sources mounted on a frame with a color camera.

In FIG. 1 the object illumination and image capture system 10, alsoreferred to herein as a “scanning system”, includes an outer frame 12Aand an inner frame 12B. A camera, such as a color camera 100, is mountedon the inner frame 12B, and a plurality, e.g., five, halogen lightsources 210, 220, 223, 240 and 250 are mounted on the outer frame. Alaser scanning device 150 for capturing object shape information may beprovided as well. Representative dimensions are a height (H) of 50 cm,and a width (W) of 100 cm.

The goniometric (i.e., directional) distribution of the lights 210–250,and their locations with respect to the camera 100, are determined apriori. Nominal goniometric data is typically provided by the lightsource manufacturer, but vary from source to source and over time.Exemplary light distribution is illustrated in FIGS. 2A and 2B for idealand real light sources, respectively. The location of the light sourcesis measured in the context of a particular data acquisition system.

An example of a scanning method that uses small light bulbs withcalibrated position and known directional distribution for photometricstereo can be found in R. Woodham, “Photometric method for determiningsurface orientation from multiple images”, Optical Engineering,19(1):139–144, 1980. One application of photometric stereo is in theinspection of surfaces to detect flaws or cracks (see, for example, M.Smith and L. Smith, “Advances in machine vision and the visualinspection of engineered and textured surfaces”, Business Briefing:Global Photonic Applications and Technology, World Markets ResearchCenter, pages 81–84, 2001). Another application of photometric stereo isthe recovery of finger prints: G. McGunnigle and M. J. Chantler,“Recovery of fingerprints using photometric stereo”, in IMVIP2001, IrishMachine Vision and Image Processing Conference, pages 192–199, September2001. Capturing images of objects illuminated by small light bulbs withcalibrated position with a geometrically calibrated camera is also usedto compute the surface properties (color and specularity) of objects foruse in computer graphics rendering systems. A summary of such methodscan be found in F. Bernardini and H. Rushmeier, “The 3d modelacquisition pipeline”, Computer Graphics Forum, 21(2), 2002. Recentpublications that describe this type of application in more detail are:Hendrik P. A. Lensch, Jan Kautz, Michael Goesele, Wolfgang Heidrich andHans-Peter Seidel, “Image-based reconstruction of spatially varyingmaterials”, in Rendering Techniques '01, London, UK, June 2001, and H.Rushmeier and F. Bernardini, “Computing consistent surface normals andcolors from photometric data”, in Proc. of the Second Intl. Conf. on 3-DDigital Imaging and Modeling, Ottawa, Canada, October 1999. The computergraphics rendering of captured objects is used in many applications,such as feature films, games, electronic retail and recording images ofcultural heritage.

Various techniques have been employed in the past to address the problemof measuring either the position of a light source or its directionaldistribution. One method for measuring the position of small lightsources is to use a separate digitizing system, such as a robotic arm orother portable coordinate measurement system. An example of such an armis described in U.S. Pat. No. 5,611,147, “Three Dimensional CoordinateMeasuring Apparatus”, Raab. Another method for measuring light sourceposition is to observe two or more shadows of objects, with knowndimensions, on a plane, where for each the coordinates are known interms of the camera coordinate system. Such a method is described inU.S. Pat. No. 6,219,063, “3D Rendering”, Bouguet et al. Knowing theposition of the base of two objects, and the end of their shadow (or thesame object in two locations), the light source position can be computedby finding the intersection of the rays joining each end-of-shadow andobject tip pair.

An ideal light source emits light isotopically, as shown in FIG. 2A.However, all practical (real) light sources emit light with adirectional distribution, as shown in FIG. 2B. A number of complexmethods have been devised to measure the directional distribution from alight source, and a summary of such methods can be found in U.S. Pat.No. 5,253,036 “Near-Field Photometric Method and Apparatus”, Ashdown.These methods generally involve taking numerous individual measurementswith a light meter in the sphere of directions surrounding the lightsource.

A robotic arm (as in U.S. Pat. No. 5,611,147), or a portable coordinatemeasurement system, can very accurately measure light source position.However, a robotic arm can be very costly, particularly if a largeworkspace (the distance between the light sources used) is considered. Arobotic arm also requires substantial human intervention, since the armtip has to be manually placed at the center of each light source to makethe measurements. Finally, after finding the light source positions, arobotic arm cannot be used to make measurements of the light sourcedirectional distribution. A separate measurement technique is requiredto make the measurement of directional distribution.

The shadow casting method described in U.S. Pat. No. 6,219,063 islimited in the space in which light positions can be calibrated. Themethod requires that the location of the plane on which the shadows arecast is known a priori. However, the location of the plane can only beknown in camera coordinates if that plane itself is used for theoriginal camera calibration. This limits the orientations of shadowsthat can be observed. Furthermore, a planar calibration method requiresthat the plane used cannot be perpendicular to the line-of-sight of thecamera. Also, the method described in U.S. Pat. No. 6,219,063 results inonly approximate light locations, as there is no method to specify theprecise location of the tip of the shadow casting object. The tip of theshadow-casting object is specified only to the accuracy of the width ofthe shadow-casting object. Since there is no unique feature of theobject base, a point on the base must be specified manually by the user.The result is also prone to error since the shadow tip is not welldefined, and can only be reliably located by the user manually locatinga pixel that coincides with the tip of the shadow-casting object. Thistechnique also requires two or more images for each light source, and nomethod for computing the light source distribution in the scanning areais included.

Determining the light source directional distribution with a series oflight meter measurements as described in U.S. Pat. No. 5,253,036 is timeconsuming, and furthermore requires the use of an additional devicebeyond the camera and light sources needed in the photometric or 3Dscanning system.

As can be appreciated, the foregoing prior art techniques for lightsource calibration are not optimum, as they involve increased cost andcomplexity, and/or manual user intervention which can give rise touser-introduced errors.

SUMMARY OF THE PREFERRED EMBODIMENTS

The foregoing and other problems are overcome, and other advantages arerealized, in accordance with the presently preferred embodiments ofthese teachings.

Disclosed herein is a system and method for calibrating light sourcesused in machine vision and computer graphics capture systems. Morespecifically, this invention pertains to a system and method forcalibrating light sources used in image capture systems where objectsare illuminated by only one point source of light at a time, and theimages of the objects are acquired with a geometrically calibratedcamera. The method employs the use of a calibration target that casts ashadow onto itself. Images of the calibration target are acquired underthe same illumination conditions as those used with actual objects. Inthat the geometries of the camera and of the target are known, theacquired images are used to geometrically characterize the cast shadowsof the target, and thus deduce the locations of the light sources. Inthe preferred embodiment the target surface also has a uniform, diffuse,light dispersive Lambertian coating. By observing the distribution oflight reflected from the target surface from each light source thedirectional distribution of the light sources can be measured.

This invention provides a system and a method for the calibration of theposition and directional distribution of light sources using only imagesof a predetermined target object. The invention assumes that the camerahas previously been geometrically calibrated. The technique fordetermining light source position builds on the simple observation thata point source of light, a point on an object and the correspondingpoint in its cast shadow all lie on the same line. The position of alight source is found by determining the intersection of a plurality ofsuch lines. Given the position of the light source, a description of itsdirectional distribution can be obtained by comparing observations oflight emitted from the source to the light that would be emitted from anideal source. In the presently preferred embodiment of this inventionthe calibration procedure is completely automatic, requiring the useronly to physically place the target object in front of the camera.

The light source location calibration procedure of this invention usesshadows cast by target objects. However, unlike the prior art approachesthat also use object shadows, there is no restriction on the location ofthe plane on which the shadows are cast, and in particular the plane maybe perpendicular to the camera line of sight. A unique target with aninscribed pattern and with geometric objects affixed to it is used, andthe location of the objects and the shadows they cast are foundprecisely without requiring user intervention, and thus withoutintroducing the possibility for user error.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other aspects of these teachings are made more evidentin the following Detailed Description of the Preferred Embodiments, whenread in conjunction with the attached Drawing Figures, wherein:

FIG. 1 is a simplified front view of a prior art scanning system thatuses an array of small light sources and a camera to capture geometricand photometric data;

FIGS. 2A and 2B illustrate light source directional distributions, wherethe ideal light source of FIG. 2A emits light uniformly in alldirections, while the light emitted from an actual light source (FIG.2B) varies with emission direction;

FIG. 3 is a front view of a white, Lambertian (diffuse) calibrationtarget used for light source calibration, the target including a blackoutline rectangle and a set of cones mounted on the diffuse surface ofthe target;

FIG. 4 shows the calibration target placed in front of a calibratedcamera surrounded by an array of small light sources;

FIG. 5 shows the calibration target placed in front of the calibratedcamera surrounded by an array of small light sources, and further showsthe position of the camera calibrated with respect to a geometricscanning device;

FIG. 6 illustrates the geometry of the intersection of lines throughpoints and the shadows they cast, for determining light source position;

FIGS. 7A–7D, collectively referred to as FIG. 7, illustrates thedetection of the target corner points and shadow tips for a set ofimages.

FIG. 8 illustrates how the reflected light intensity is observed atpoints distributed across the light calibration target, where thereflected intensity is compared to the intensity that would be reflectedfrom an ideal light source;

FIGS. 9A and 9B, collectively referred to as FIG. 9, illustrate thatsampling points across the target surface is equivalent to samplingdirections from the light source;

FIG. 10 is a logic flow diagram that illustrates a light sourcecalibration method in accordance with this invention;

FIG. 11 is a logic flow diagram that illustrates in further detail theprocess image step in FIG. 10; and

FIG. 12 is a depiction of one suitable calibration target or object forcalibrating the digital camera.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Disclosed herein is light source calibration technique for calibratingan image capture system working volume, such as the image capture system10 shown in FIG. 1, that is arbitrarily oriented with respect to thecamera 100 of the photometric system. The light source calibrationtechnique uses the camera 100 to acquire a set of digital images of aspecially designed calibration target, with one image taken with eachone of the light sources 210–250 to be calibrated (i.e., one lightsource is turned on at a time, and an image is captured for eachillumination source in turn). The only required manual intervention isto physically place the calibration target in front of the camera 100.The photometric or scanning system 10 requires a geometricallycalibrated camera 100. By “geometrically calibrated” what is implied isthat, in terms of a 3D coordinate system in the world, the extrinsiccamera properties of camera location, view direction, and “up”orientation are known. These extrinsic properties can be expressed as atranslation T and a rotation R, expressed in the world coordinatesystem, that enable any object in the world system to be transformed tothe camera coordinate system, where the “eye” or entrance pupil 100A ofthe camera 100 is located at the origin (0,0,0). By “geometricallycalibrated” what is also implied is that the focal length of the camera100 has been measured. The geometric calibration of the camera 100 canbe performed by any of a variety of known techniques. In the preferredembodiment of the calibration system a non-coplanar camera calibrationmethod described by Roger Y. Tsai, “An efficient and accurate cameracalibration technique for 3D machine vision, in Computer Vision andPattern Recognition, pages 364–374, June 1986, is used, although theteachings of this invention are not limited to only this one particularcamera calibration technique.

FIG. 12 shows an example of a calibration target 1000 for the camera 100alignment, in this case a cube with a checkerboard pattern for camera100 and laser scanner calibration 900. The calibration procedure of Tsaiis used with the cube having a precise checkerboard pattern on each ofthree adjacent sides to establish the three dimensional positions of thetwo dimensional features detected in the image of the cube. It ispreferred for many applications of interest that sub-millimeter accuracyis achieved so that when scans are registered and combined with thecolors, the fine-detailed normals are not blurred. Such accuracyrequires a calibration target 1000 with the corners placed with anaccuracy of 0.1 mm over a distance of about 20 cm. A variety ofpatterning techniques, including printing on various media and theprecision milling polymeric or metallic materials can be employed. Tomaintain the required accuracy with a surface that is not specular, thesurface may be created by mechanically sandblasting 6061 Al plates tocreate a random surface texture, combined with a chemical etch to smoothmicroscopic specular regions. The checkerboard pattern can be exposed ina dry film photo-resist with a step and repeat camera to maintain tightcumulative geometric tolerance across the large substrate. Exposedregions in the photo-resist are subsequently black anodized.

The calibration begins with the acquisition of the cube in a standardposition, as shown in FIG. 12. The corners of the three checkerboardsare then detected for providing camera 100 calibration data. Thepreferred camera 100 calibration method employs several fully automaticphases during which patterns of increasing complexity are detected. But,depending on the visual feedback, the user decides at which phase tobegin and provides input clicks accordingly. The full process is appliedindependently to each checkerboard. Each phase begins with a knownpattern defined by a certain number of points. In phase 1, the userclicks a triangle (a, b, c) representing a normal basis of thecheckerboard. Then, from the extrapolated parallelogram (a, b, d, c),and using an automatic algorithm, each of the four corners isautomatically adjusted so that the quadrangle matches “perfectly” thecorresponding image edges. This is done by maximizing the integral ofthe magnitude of the image intensity gradient along the edges of thequadrangle pattern. The integral can be maximized by using a gradientascent algorithm in multiple image scales to avoid irrelevant localminima. The initial parallelogram is allowed to deform as a generalquadrangle because the camera 100 projection does not preserveparallelism. Each additional phase starts either using the known cornersresulting from the previous phase or, if the user selects, by the samecorners but specified manually. During the second phase the knownquadrangle is extended by linear extrapolation to form the quadrangle(a, e, f, g). Using the same optimization algorithm, these four cornersare then fitted automatically to the image data. The corner h is theninitialized by quadratic extrapolation using a, b and e, accounting tosome extent for projective depth distortion. The corners i and j areinitialized similarly and the quadrangle (a, h, i, j) is automaticallyfitted. This process is repeated until all the corners (a, b, e, h, . .. , k), (a, d, f, i, . . . , l) and (a, c, g, j, . . . , m) are found.During the third phase, using linear, and whenever possible quadraticextrapolations, followed by automatic fitting to image data, eventuallyall the corners of the boundary of the checkerboard are detected. It ispossible to do so by only considering a quadrangle with edges at leastas large as half of the full checkerboard side. Avoiding shorter edgeshas been found to increase robustness. During the fourth phase, andstarting from the known boundary corners, all the inside corners areinitialized by simply intersecting “horizontal” and “vertical” lines.Due to camera 100 lens radial distortion, the “lines” are actuallyslightly curved, and deviations from the ideal line on the order ofthree pixels can be observed. This is accounted for during the fifthphase by fitting individual cross-hair patterns for each corner. Thecross-hair pattern (s, t, u, v) is initialized and automatically fitted.The only allowed degrees of freedom during the optimization are thetranslation and the angle between su and tv. Furthermore, for eachcross-hair pattern, this algorithm is repeated a number of times,starting from randomly selected nearby initial guesses. Thecorresponding corner is only considered as successfully detected if allof these tries yield the same result, up to some fraction e.g., onetenth, of a pixel. This is one suitable technique to enforce thesignificance of the detected corner. Finally, the detected corners aredisplayed, superimposed with the checkerboard image. The user thenaccepts or rejects the achieved detection, thereby completing the cameracalibration procedure.

Referring to FIG. 3, an important aspect of this invention is the lightsource calibration target 20. The light source calibration target 20includes a planar surface 300 that is preferably white and diffuselyreflecting (Lambertian or substantially Lambertian) on which shadows canbe cast. To produce the white diffuse target surface a conventionalmatte finish white paint can be used to coat a suitable smooth substratematerial, or a specialized material such as LabSphere Spectralon™ may beused to coat the substrate material. The target 20 includes a set ofwell defined point features, such as the corners of a polygon 400 drawnin a visually contrasting color, e.g., black, on the surface 300. Thetarget 20 also includes a plurality of 3D geometric objects affixed tothe surface 300. In the case of FIG. 3 the plurality of objects areembodied as four cones 500 with bases mounted to the surface 300 andwith sharp tips located above the surface 300. A set of exemplaryshadows 600 cast by the cones 500 are shown for an assumed light sourcelocated above and to the right of the surface 300.

It is possible to compute the position of the surface 300 with respectto the camera 100 by marking a number of points on the surface 300. Ingeneral, a set of more than three points on a plane with known relativelocations can be used to determine the location of the plane withrespect to a geometrically calibrated camera, knowing only thecoordinates of the point projections in the camera image. Reference inthis regard may be had to B. Triggs, “Camera pose and calibration from 4or 5 known 3D points”, in ICCV 99, pp. 278–284, Kerkyra, Greece,September 1999.

Consider a calibrated camera 100, three known points and the imagelocations these points. Knowing the image locations the point locationsare known to be located along the rays starting at the camera origin andpassing through the image plane. Knowing the relative distance betweenthe points completely specifies where along these rays the 3D points liein terms of the calibrated camera coordinate system. Given the locationof the points, the location of the plane is readily calculated in thecamera coordinate system. The location of any objects fixed to the planein a known fashion is then also known.

From an image processing standpoint, the pixel coordinates of knownpoints on the target surface 300 are better defined as the intersectionsof lines seen in the image. Lines themselves are better defined asinterfaces between bright and dark areas. One technique of marking fourpoints on the plane reconciling these requirements is to draw in avisually contrasting manner a square or a rectangle as the polygon 400.The corners of the polygon 400 thereby form the set of reference points.

Additionally, the 3D features are placed between this polygon 400 andthe light sources 210–250 in order to cast shadows. These 3D featuresare preferably the cones 500 mounted on the surface 300 and facing thecamera 100, as shown in FIGS. 3, 4 and 5. FIG. 4 shows the calibrationtarget surface 300 placed in front of the calibrated camera 100,surrounded by the array 200 of small light sources 210–250, while FIG. 5also shows an alternative embodiment where he position of the camera iscalibrated with respect to a geometric scanning device 900, such as thelaser scanner 150 shown in FIG. 1.

The positions of the tips of the cones 500 with respect to the camera100 are known, as the geometry of the target 20 is known, including thecorners of the polygon 400. Each cone 500 casts a conic shadow 600 onthe white planar surface 300. The tip of the shadow 600 can be locatedaccurately in the camera image since it is defined as the intersectionof two lines which are themselves defined as the interface between darkand bright areas. Knowing the location of the plane and the pixelcoordinates of the tip of the shadow 600 is sufficient to compute the 3Dlocation of the tip of the shadow 600. Then, since the 3D locations ofthe tips of the cones 500, and of the shadows 600, are both known, onecan determine that the energized light source 210-250 is located on theline formed by these two points. Using at least two such cones 500mounted within the white square 400, one is thus able to determine thelocation of the light source 210–250 as the intersection of these lines.The underlying geometry of the intersection of lines through points andthe shadows they cast, for determining the light source position, isillustrated in FIG. 6. In FIG. 6 the line1 connects the tip of cone1 andthe tip of its shadow (shadow1). Line2 connects the tip of cone2 and tipof its shadow (shadow2). The intersection of lines 1 and 2 can be seento be coincident with the position of the light source.

The intersection of the lines for the two or more cast shadows can becomputed by a non-linear least square method. One suitable non-linearleast squares method is known as the Levenberg-Marquardt method, (see,for example, Burton S. Garbow, Kenneth E. Hillstrom and Jorge J. More,“Documentation for MINPACK subroutine LMDIF1 Double Precision Version”,Argonne National Laboratory, March 1980).

FIG. 7 illustrates the detection of the target polygon 400 corner pointsand the shadow 600 tips for a set of images. For each image thepreferred embodiment applies a common image edge detection operator tothe image. Lines are fit through sets of pixels that form approximatelystraight lines in the image. The intersections of the outmost sets oflines are identified as the corners of the polygon 400, and allow thelocation and orientation of the target surface 300 to be computed in thecamera 100 coordinate system. Knowing the orientation of the surface300, the locations of the cones 500 in the image are known and can beblocked out. The remaining short intersecting lines by each cone 500location are identified as the edges of the cone shadow 600. Theintersection of the shadow edges are identified then as the tip of thecone shadow 600. For the illustrated example, FIG. 7A shows the capturedimage, FIG. 7B shows the result of applying edge detection to find theoutline and corners of the polygon 400 and the cone 600 tips, FIG. 7Cshows the result of fitting lines to identify points on corners of thepolygon 400 used to compute the location of the target 20, and FIG. 7Dshows the result of fitting lines to identify the tips of the shadows600. It should be noted that the width of lines and point locations havebeen exaggerated for visual clarity in FIGS. 7C and 7D.

The diffuse nature of the target surface 300 facilitates the estimate ofthe variation of the light source distribution from the ideal isotropicdistribution. As shown in FIG. 8, the light magnitude of the light fordifferent directions from a light source (e.g., 210) is sampled byobserving the light reflected from the target surface 300 at pointsdistributed across the light source calibration target 20. The sample oflight reflected at the selected points may be taken as the pixel valueat that point, or to reduce the effect of noise in the image, an averageof values around the location may be used. For an ideal isotropic lightsource, the reflected light is given by the light source intensity timesthe cosine of the angle between a ray from the point to the light sourceand the surface normal, divided by the distance to the source squared.By computing how the observed reflected light varies from this idealreflection, a ratio is formed characterizing the light in eachdirection, as shown in a 2D example in FIG. 9. For an application whenall objects are to be scanned on the plane (the case shown in FIG. 9A),the correction for non-ideal light source distribution is computed by asimple interpolation. For an application to 3D objects, the pointobserved by the camera is identified in 3D, and the direction from thelight source to the point is computed, as shown in FIG. 9B. Thecorrection for the non-ideal distribution then is found by interpolatingthe corrections for the neighboring directions.

More specifically, FIG. 9 shows that sampling points across the targetsurface 300 is equivalent to sampling directions from the source. InFIG. 9A a ratio of real (Im) to ideal light (Ic) intensity for eachsample direction can be computed as a light correction factor. In thisexample values of 0.96, 0.85 and 0.75 are found. The light intensity forother directions can be found then by interpolation, for either the caseof scanning on a plane similar to the target 300 (FIG. 9A) or a true 3Dobject (FIG. 9B). In FIG. 9A, the incident light is computed at point Sas the light from the ideal source divided by the correction factor thatis interpolated between the values of 0.75 and 0.85. In FIG. 9B theincident light at point Q is computed as the light from the ideal sourcedivided by the correction factor that is interpolated between 0.85 and0.96.

In the preferred embodiment of this invention the positions of the lightsources 210, 220, 230, 240 and 250 are obtained by taking an image ofthe calibration surface 300, which has been coated with a diffuselyreflecting material (e.g., white paint), with each of the lights sources210–250 turned on in turn. The polygonal shape 400 with sharp cornershas been inscribed on the surface 300, and the set of cones 500 havebeen mounted on the surface 300. When one of the array 200 of lights210–250 is illuminated, the set of shadows 600 are cast by the cones500. The use of the system shown in FIG. 4 is illustrated in FIG. 10. Itis assumed that a data processor 180 is coupled to the camera 100 andreceives captured images therefrom. The data processor 180 is programmedto implement the following method.

In the first step 700 the camera 100 is calibrated using any suitablemethod, such as the method described by Tsai. In step 710 thecalibration target 20 having surface 300 is placed in view of the camera100. For each of the light sources 210–250 to be calibrated, an image iscaptured in step 720, and then processed in step 730. In the final stepthe calibrated light source data is stored 800. The target 20 can thenbe removed, and an object to be imaged placed in front of the camera100.

FIG. 11 shows the details of the image processing performed by the dataprocessor 180 in step 730 of FIG. 10. In step 733 the corners of thepolygonal shape 400 are detected. This may be done by applying edgedetection and line fitting as illustrated in FIGS. 7B and 7C. In step736 the location of the surface 300 in terms of the coordinate system ofthe calibrated camera 100 is calculated using the image locationsdetected, and the known distances between the 3D points on the surface300. In step 739 the tips of the shadows 600 are detected as illustratedFIG. 7D. In step 742 the light source positions are computed from theknown locations of the tips of the cones 500 and cone shadows 600. Thiscalculation preferably finds a least squares solution for theintersection of all of the lines formed by the shadow 600 tips andcorresponding cone 500 tips in the image. If the relative positions ofthe light sources 210–250 is fixed, the calculation can be made moreaccurately by solving for all of the light source positionssimultaneously. In step 745 the intensities of the pixels in the imageacquired in step 720 are sampled at some preset number of points on thetarget (as shown in FIG. 8), and are compared to the intensities thatwould be observed in the presence of an ideal isotropic light source. Instep 748 a directional distribution for each light source 210–250 iscomputed as an interpolation function between the corrected light valuesfound at the sample points. When multiple sources are being processed,steps 733 and 736 only need to be performed for the first source. Thelocation of the target surface 300 is then known and can be used in thecalculations for the subsequent sources.

FIG. 5 shows an alternative embodiment, in which the separate shapecapture device 900 is used with the calibrated camera 100 and lightsource array 200. The shape captured with device 900 is calibrated inthe same coordinate system as camera 100, so that surface detail fromimages captured by camera 100 using the light source array 200 can becomputed for shapes captured with device 900.

In one suitable embodiment the system 10 includes a ShapeGrabber™ laserrange scanner 900, a Fuji FinePix™ S1 Pro digital color camera 100, andfive halogen (150 W) light sources 210–250 mounted on a 100 cm by 50 cmaluminum rack, as shown in FIG. 1. The system may be used with ascanning table that contains an embedded Kaidan turntable. The use of ascanning table with embedded turntable enables scanning with littlemanual repositioning of the scanner 900 relative to the object. Thecamera 100 preferably has an adjustable lens so that the user can adjustthe zoom to accommodate scanning objects of various sizes. The camera100 can also be moved and tilted with respect to the range scanner 900to adjust for different types of objects. The halogen bulbs are enclosedin a housing that includes a diffuser in front of the bulb to ensuresmooth directional distribution of emitted light. All of the systemelements are under control from a scanning workstation that embodies thedata processor 180, such as an IBM Intellistation™ MPro™ with a Pentium™IV microprocessor and NVIDIA Quadro2™ graphics. The turntable, a lightsource relay box and a laser scanner 900 panning stage are allcontrolled through serial connections. The color camera 100 is connectedto the data processor 180 via a USB port. Communication to the scanninghead is by IEEE 1394 (fire wire.)

Various suitable and exemplary dimensions for the embodiments depictedin the Figures are as follows: the target surface 300 may be 60 cm by 40cm; the polygon 400 may be a square 25 cm on a side, and thecenter-to-center spacing between the cones 500 may be 15 cm. Each cone500 may have a circular base of diameter 1 cm and a height of 4 cm. InFIGS. 4 and 5 the distance D may be 60 cm. In FIG. 12 the cube 1000 maybe 20 cm on a side, and each checkerboard square may be 2 cm on a side.

One implementation of the light source calibration procedure is in ascanner used for cultural heritage applications, such as one forscanning and recording the shapes of archeological artifacts. Inaddition, the teachings of this invention can be used in a number ofdifferent types of applications, including as examples only, medicalapplications to image various objects, fingerprint applications, as wellas for industrial 3D inspection applications (e.g., for crack and flawdetection) and for the imaging of objects for advertising purposes, suchas for web-enabled electronic commerce applications. The teachings ofthis invention are also not limited to the use of visual light sourcesand detectors (cameras), but may be used with other frequencies oflight, including infrared and ultraviolet light, with suitabledetector(s).

The foregoing description has provided by way of exemplary andnon-limiting examples a full and informative description of the bestmethod and apparatus presently contemplated by the inventor for carryingout the invention. However, various modifications and adaptations maybecome apparent to those skilled in the relevant arts in view of theforegoing description, when read in conjunction with the accompanyingdrawings and the appended claims.

As but some examples, the use of a single light source that istranslated from position to position can be used in lieu of a pluralityof light sources at fixed positions. As another example, other 3D objectshapes than cones having circular cylindrical base can be used for thecalibration objects 500, such as objects having a polyhedral, e.g., arectangular or a square or a hexagonal, base shape and that terminate ina point or some other feature that casts a shadow that can be readilyand accurately located in the captured image. Also, other than a whitecoating material can be used, so long as sufficient distinction in thecaptured images can be made between the background (surface 300) and thereference points, e.g., the corners of the polygon 400, as well as thetips of the cones 600 and the cast shadows 700.

However, all such and similar modifications of the teachings of thisinvention will still fall within the scope of this invention. Further,while the method and apparatus described herein are provided with acertain degree of specificity, the present invention could beimplemented with either greater or lesser specificity, depending on theneeds of the user. Further, some of the features of the presentinvention could be used to advantage without the corresponding use ofother features. As such, the foregoing description should be consideredas merely illustrative of the principles of the present invention, andnot in limitation thereof, as this invention is defined by the claimswhich follow.

1. A light source calibration target, comprising: a substrate having asubstantially Lambertian surface; a plurality of reference marks formedon said surface that are visually distinct from said surface andcomprising at least one visually distinct shape having corners formed onsaid surface so as to be visually distinct from said surface; and aplurality of three dimensional objects mounted on said surface eachhaving an upstanding feature disposed above said surface; said lightsource calibration target for being disposed in a field of view of animage capture system that includes a camera to calibrate individual onesof a plurality of light sources by operating the camera to capture animage of the light source calibration target with an individual one ofthe light sources turned on, by constructing a plurality of lines eachof which intersects the feature of an object and a point on a shadowcast by the feature, and by determining a location of the light sourceas a point of intersection of the plurality of lines, and furthercomprising determining a correction factor corresponding to a differencebetween a light source emission distribution and an ideal light sourceemission distribution.
 2. A light source calibration target as in claim1, where said reference marks comprise corners of a polygonal pattern.3. A light source calibration target as in claim 1, where said pluralityof three dimensional objects comprise cone shaped objects having a basemounted to said surface and upstanding tips.
 4. A light sourcecalibration target as in claim 1, where said reference marks comprisecorners of a polygonal pattern, and where said plurality of threedimensional objects comprise at least two cone shaped objects having abase mounted to said surface within said polygonal pattern.
 5. A methodto calibrate a plurality of light sources of an image capture systemthat includes a calibrated camera, comprising: placing a calibrationtarget having a surface in view of the camera, the target comprising asubstrate having a substantially Lambertian surface, a visually distinctpolygonal shape having corners formed on said surface so as to bevisually distinct from said surface, and a plurality of objects eachhaving an upstanding tip mounted on said surface; for each of the lightsources to be calibrated, capturing an image of the target; processingthe captured image to derive light source calibration data; and storingthe calibration data, where processing the captured image comprises:detecting the locations of the corners of the polygonal shape in thecaptured image; determining the location of the surface in terms of acoordinate system of the calibrated camera using the corner locationsand known distances between three dimensional points on the surface; foreach of the light sources to be calibrated, detecting the locations ofthe tips of shadows cast by the plurality of objects; determining thelight source position from the known locations of the tips of theplurality of objects and from the detected locations of the tips of theshadows by an intersection of lines formed by the shadow tips andcorresponding object tips in the captured image; sampling intensities ofimage pixels at sample points and comparing the sampled intensities tointensities that would be observed in if the light source were an idealisotropic light source to determine corrected values; and computing adirectional distribution for the light source using the correctedvalues.
 6. A method as in claim 5, where computing a directionaldistribution comprises interpolating.
 7. An image capture systemcomprising a plurality of light sources, a calibrated digital camera anda data processor coupled to the camera, further comprising: acalibration target having a surface in view of the camera, the targetcomprising a substrate having a substantially Lambertian surface, avisually distinct polygonal shape having corners formed on said surfaceso as to be visually distinct from said surface, and a plurality ofobjects each having an upstanding tip mounted on said surface; and saiddata processor operating with said camera under control of a storedprogram, for each of the light sources to be calibrated, for capturingan image of the target and processing the captured image to derive lightsource calibration data and for storing the calibration data, where saiddata processor, when processing the captured image, operates undercontrol of said stored program for detecting the locations of thecorners of the polygonal shape in the captured image; determining thelocation of the surface in terms of a coordinate system of thecalibrated camera using the corner locations and known distances betweenthree dimensional points on the surface; for each of the light sourcesto be calibrated, detecting the locations of the tips of shadows cast bythe plurality of objects; determining the light source position from theknown locations of the tips of the plurality of objects and from thedetected locations of the tips of the shadows by an intersection oflines formed by the shadow tips and corresponding object tips in thecaptured image; sampling intensities of image pixels at sample pointsand comparing the sampled intensities to intensities that would beobserved in if the light source were an ideal isotropic light source todetermine corrected values; and computing a directional distribution forthe light source using the corrected values.
 8. A system as in claim 7,where computing a directional distribution comprises interpolating.
 9. Amethod to calibrate individual ones of a plurality of light sources ofan image capture system that includes a camera, comprising: placing acalibration target having a surface in view of the camera, the targetcomprising a substrate having a diffusely reflecting surface, a visuallydistinct shape having corners formed on said surface so as to bevisually distinct from said surface, and a plurality of objects mountedon said surface each having an upstanding feature disposed above saidsurface; for each of the light sources to be calibrated, capturing animage of the target with the light source turned on and constructing aplurality of lines each of which intersects the feature of an object anda point on a shadow cast by the feature; and determining a location ofthe light source as a point of intersection of the plurality of lines,further comprising determining a correction factor corresponding to adifference between a light source emission distribution and an ideallight source emission distribution.
 10. A method as in claim 9, furthercomprising initially processing a captured image to determine anorientation of the surface relative to the camera based on an image ofthe corners.
 11. A method as in claim 9, further comprising initiallycalibrating the camera.
 12. A computer program product embodied on or ina computer readable media, comprising program instructions encoded forcausing a computer to perform a method to calibrate individual ones of aplurality of light sources of an image capture system that includes acamera, where a calibration target having a surface is placed in view ofthe camera, the target comprising a substrate having a diffuselyreflecting surface, a visually distinct shape having corners formed onsaid surface so as to be visually distinct from said surface, and aplurality of objects mounted on said surface each having an upstandingfeature disposed above said surface, the method operating, for each ofthe light sources to be calibrated, to capture an image of the targetwith the light source turned on, to construct a plurality of lines eachof which intersects the feature of an object and a point on a shadowcast by the feature, and to determine a location of the light source asa point of intersection of the plurality of lines, where the methodfurther comprises determining a correction factor corresponding to adifference between a light source emission distribution and an ideallight source emission distribution.
 13. A computer program product as inclaim 12, where the method further comprises initially calibrating thecamera.
 14. A computer program product embodied on or in a computerreadable media, comprising program instructions encoded for causing acomputer to perform a method to calibrate individual ones of a pluralityof light sources of an image capture system that includes a camera,where a calibration target having a surface is placed in view of thecamera, the target comprising a substrate having a diffusely reflectingsurface, a visually distinct shape having corners formed on said surfaceso as to be visually distinct from said surface, and a plurality ofobjects mounted on said surface each having an upstanding featuredisposed above said surface, the method operating, for each of the lightsources to be calibrated, to capture an image of the target with thelight source turned on, to construct a plurality of lines each of whichintersects the feature of an object and a point on a shadow cast by thefeature, and to determine a location of the light source as a point ofintersection of the plurality of lines, where the method furthercomprises initially processing a captured image to determine anorientation of the surface relative to the camera based on an image ofthe corners.